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Abstract. In this paper we prove that the symmetric pair (GL n+ i c (F), GL n (F) xGLjj(F)) is a Gelfand 
pair for any local field F of characteristic 0. For non-archimedean F it has been proven in |JRI . We use 
OO i techniques developed in AG2 to generalize their proof to general local fields. 
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+± ■ 1. Introduction 

Fix a local field F of characteristic zero. Consider the standard embedding of GL n x GLk to GL n+ j-- 
The goal of this paper is to prove the following theorem. 



Theorem (I). The symmetric pair {GLyi-^k, GL n x GLk ) is a Gelfand pair. Namely, for any irreducible 
\ admissible representation (tt,E) of GL n +k{F) we have 

(1) dimHom GLn{ F)xGL h (F){E°°,C) < 1. 

m ■ 

Using the classical arguments of Gelfand and Kazhdan we deduce it (in section l2~Tj) from the following 
. theorem. 

qq ' Theorem (II). Any distribution £ on GL n+ k(F) which is invariant with respect to two-sided action of 

GL n {F) x GLk(F) is invariant with respect to the antiinvolution g i— » g^ 1 . 

In the non-archimedean case these theorems had been proven in [JR.] . The techniques developed in 
|AG2j enabled us to generalize that proof to work in the archimedean case. Also, those techniques made 
■ the proof shorter by eliminating some of the technical computations. 

In the case k = 1, already the pair (GL n x {l},GL n+ i) is a Gelfand pair (see [AGSj ). For non- 
archimedean F it is even proven to be a strong Gelfand pair (see [AGRSj l. 

Structure of the paper. In subsection 2.1 we give preliminaries on Gelfand pairs and their connection 
to invariant distributions. In subsections 2.2-2.4 we introduce those techniques developed in |AG2j which 
are relevant to symmetric pairs. We also formulate theorem IF which is equivalent to theorem II. In 
section 3 we prove theorem IF. 
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2. Preliminaries and notations 

• All the algebraic varieties and algebraic groups that we will consider will be defined over F. 

• For a group G acting on a set X and an element x £ X we denote by G x the stabilizer of x. 

• By a reductive group we mean an algebraic reductive group. 

In this paper we will refer to distributions on algebraic varieties over archimedean and non-archimedean 
fields. In the non-archimedean case we mean the notion of distributions on ^-spaces from |BZJ . that is 
linear Junctionals on the space of locally constant compactly supported functions. In the archimedean 
case one can consider the usual notion of distributions, that is continuous functionals on the space of 
smooth compactly supported functions, or the notion of Schwartz distributions (see e.g. |AGlj V It does 
not matter here which notion to choose since in the cases of consideration of this paper, if there are no 
nonzero equivariant Schwartz distributions then there are no nonzero equivariant distributions at all (see 
Theorem 4.0.8 in [AT?2] ). 

2.1. Gelfand pairs. 

In this section we recall a technique due to Gelfand and Kazhdan ( |GK| ) which allows to deduce statements 
in representation theory from statements on invariant distributions. For more detailed description see 
[A"GS] . section 2. 

Definition 2.1.1. Let G be a reductive group. By an admissible representation of G we mean an 
admissible representation of G(F) if F is non-archimedean (see [BZj ) and admissible smooth Frechet 
representation of G(F) if F is archimedean. 

We now introduce three notions of Gelfand pair. 

Definition 2.1.2. Let H C G be a pair of reductive groups. 

• We say that (G,H) satisfy GP1 if for any irreducible admissible representation (~k,E) of G we 
have 

dim Horn H(p)(E,C) < 1 

• We say that (G,H) satisfy GP2 if for any irreducible admissible representation (tt,E) of G we 
have 

dim Hom H ( F ) (E, C) ■ dim Horn h{E,<C) < 1 

• We say that (G,H) satisfy GP3 if for any irreducible unitary representation (tt, H.) of G{F) on 
a Hilbert space Ti. we have 

dim Hom H(F) (H°°,C) < 1- 

Property GP1 was established by Gelfand and Kazhdan in certain p-adic cases (see |GK) ) . Property 
GP2 was introduced in |Gro| in the p-adic setting. Property GP3 was studied extensively by various 
authors under the name generalized Gelfand pair both in the real and p-adic settings (see e.g. |vDi 
lyDPllBvD] ). 

We have the following straightforward proposition. 

Proposition 2.1.3. GP1 =>■ GP2 => GP'i. 

We will use the following theorem from [AGS] which is a version of a classical theorem of Gelfand and 
Kazhdan. 

Theorem 2.1.4. Let H C G be reductive groups and let r be an involutive anti- automorphism of G and 
assume that t(H) = H. Suppose t(£) = £ for all bi H(F) -invariant distributions £ on G(F). Then 
(G, H) satisfies GP2. 

In our case GP2 is equivalent to GP1 by the following proposition. 

Proposition 2.1.5. Suppose H C GL n is transpose invariant subgroup. Then GP1 is equivalent to GP2 
for the pair (GL„,iJ). 

For proof see AGSJ, proposition 2.4.1. 

Corollary 2.1.6. Theorem II implies Theorem I. 
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2.2. Symmetric pairs. 

In this subsection we review some tools developed in AQ2] that enable to prove that a symmetric pair 
is a Gelfand pair. 

Definition 2.2.1. A symmetric pair is a triple (G,H,9) where H C G are reductive groups, and 9 is 
an involution of G such that H — G B . We call a symmetric pair connected if G/H is connected. 

For a symmetric pair (G,H,9) we define an antiinvolution a : G — » G by <r(g) '■= 9(g~ r ), denote 
g := LieG, f) := LieH, g a := {a £ $\9(a) = —a}. Note that H acts on g a by the adjoint action. Denote 
also G a := {g £ G\a(g) — g} and define a symmetrization map s : G — > G a by s(g) := gcr(g). 

Definition 2.2.2. Let (G±, H±,8i) and (G2,i?2,#2) be symmetric pairs. We define their product to be 

the symmetric pair (G\ x Gi,H\ x Fli,9\ x 2 )- 

Definition 2.2.3. We call a symmetric pair (G,H,9) good if for any closed H(F) x H(F) orbit O C 
G(F), we have a(0) = O. 

Proposition 2.2.4. Let (G,H,9) be a connected symmetric pair. Suppose that for any g G G(F) with 
closed orbit, H 1 (F, (H x H) g ) is trivial. Then the pair (G,H,9) is good. 

For proof see |AG2| . Corollary 7.1.5. 

Definition 2.2.5. We say that a symmetric pair (G, H, 8) is a GK pair if any H(F) x H(F) - invariant 
distribution on G(F) is a - invariant. 

Definition 2.2.6. We define an involution 9 n ^ : GL n+ k — * GL n+ k by ni k{%) = £xe where e = 

fldnn Onk \ Note that (GL n+ k,GL n x GLk,9 n k) is a symmetric pair. If there is no ambiguity 

\ Ufe„ —-Idkk J 

we will denote 9 n ^ just by 9. 

Theorem II is equivalent to the following theorem. 

Theorem (IT). The pair (GL n+ k,GL n x GLk,9 n ^k) is a GK pair. 

2.3. Descendants of symmetric pairs. 

Proposition 2.3.1. Let (G,H,9) be a symmetric pair. Let g 6 G(F) such that HgH is closed. Let 
x = s(g). Then x is semisimple. 

For proof see e.g. |AG2j . Proposition 7.2.1. 

Definition 2.3.2. In the notations of the previous proposition we will say that the pair (G x , H x ,6\g x ) is 
a descendant of (G, H, 9). 

2.4. Regular symmetric pairs. 

Notation 2.4.1. Let V be an algebraic finite dimensional representation over F of a reductive group G. 
Denote Q(V) := V/V G . Since G is reductive, there is a canonical embedding Q(V) V . 

Notation 2.4.2. Let (G, H, 9) be a symmetric pair. We denote by Mg,h the subset of all the nilpotent 
elements in Q{g a ). Denote Rg,h '■= Q(s' 7 ) —Ng,h- 

Our notion of Rg,h coincides with the notion used in [AG 2] . Notation 2.1.10. This follows from 

Lemma 7.1.11 in [A"G1] . 

Definition 2.4.3. Let w be an action of a reductive group G on a smooth affine variety X . We say that 
an algebraic automorphism t of X is G -admissible if 

(i) ir(G(F j) is of index at most 2 in the group of automorphisms of X generated by tt(G(F)) and r. 

(ii) For any closed G(F) orbit O C X(F), we have r(O) = O. 

Definition 2.4.4. Let (G, Ft, 9) be a symmetric pair. We call an element g £ G(F) admissible if 

(i) Ad(g) commutes with 9 (or, equivalently, s(g) € Z(G)) and 

(ii) Ad(g)\ g v is H -admissible. 
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Definition 2.4.5. We call a symmetric pair (G,H,9) regular if for any admissible g G G(F) such that 
every H(F) -invariant distribution on Rg.h is also Ad(g) -invariant, we have 
(*) every H(F) -invariant distribution on Q{Q a ) is also Ad(g) -invariant. 

Clearly product of regular pairs is regular (see |AG2j . Proposition 7.4.4). 

Theorem 2.4.6. Let (G,H,9) be a good symmetric pair such that all its descendants (including itself) 
are regular. Then it is a GK pair. 

For proof see [AG2| . Theorem 7.4.5. 

Now we would like to formulate a regularity criterion. For it we will need the following lemma and 
notation. 

Lemma 2.4.7. Let (G,H,9) be a symmetric pair. Then any nilpotent element x G Q a can be extended 
to an sl2 triple (x,d(x),X-) such that d(x) G f) and X- G g a . 

For proof see e.g. |AG2] . Lemma 7.1.11. 

Notation 2.4.8. We will use the notation d{x) from the last lemma in the future. It is not uniquely 
defined but whenever we will use this notation nothing will depend on its choice. 

Proposition 2.4.9. Let (G,H,0) be a symmetric pair. Suppose that for any nilpotent x G g a we have 

Tr(ad(d(x))\ hx ) <dim CT . 

Then the pair (G, H, 9) is regular. 

This proposition follows from |AG2) (Propositions 7.3.7 and 7.3.5 and Remark 7.4.3). 

Notation 2.4.10. Let E be a quadratic extension of F. Let G be an algebraic group defined over F. We 
denote by Ge/f ^ e canonical algebraic group defined over F such that Ge/f(F) — G(E). 

Theorem 2.4.11. Let G be a reductive group. 

(i) Consider the involution 9 of G x G given by 9{{g,h)) := (h,g). Its fixed points form the diagonal 
subgroup AG. Then the symmetric pair (G x G, AG,#) is regular. 

(ii) Let E be a quadratic extension of F. Consider the involution 7 of Ge/f given by the nontrivial 
element of Gal(E/F). Its fixed points form G. Then the symmetric pair {Ge/Fj G, 7) is regular. 

This theorem follows from |AG2] (Theorem 7.6.5, Proposition 7.3.5 and Remark 7.4.3). 

3. Proof of theorem IF 

By theorem 12.4.61 it is enough to prove that our pair is good and all its descendants are regular. 
In the first subsection we will compute the descendants of our pair. Proposition 12 . 2 .41 will imply that 
our pair is good. 

In the second subsection we will prove that all the descendants are regular using theorem 12.4.111 and 
proposition 12.4.91 

3.1. Computation of descendants of the pair (GL n+ k,GL n x GL&). 

Theorem 3.1.1. All the descendants of the pair {GL n+ k,GL n x GLk,9 n ^) are products of pairs of the 
types 

(i) {{GL m ) E /F x (GL m ) E /Fi A-(GL m ) E /F,@) f or some field extension E/F 

(ii) ((GL m ) E /Fi (GL m ) L / F ,j) for some field extension L/F and its quadratic extension E/L 
(Hi) (GL m+ i,GL m x GLt,9 m ,i). 

Proof. Let x G GL° +k (F) be a semisimple element. We have to compute G x and H x . Since x G G CT , we 
have exe = x^ 1 . Let V — F n+k . Decompose V := ©* =1 V% such that the minimal polynomial of x\v t is 
irreducible. Now G X (F) decomposes to a product of GLe^i), where Ei is the extension of F defined 
by the minimal polynomial of x\vi and the -Ei-vector space structure on Vi is given by x. 

Clearly for any i, e(Vi) = Vj for some j. Now we see that V is a direct sum of spaces of the following 
two types 
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A. Wi © W2 such that the minimal polynomials of x\w t are irreducible and e(Wi) = W2. 

B. W such that the minimal polynomial of x\w is irreducible and e(W) — W. 
It is easy to see that in case A we get the symmetric pair (i). 

In case B there are two possibilities: (1) x = x~ l and (2) x ^ x^ 1 . It is easy to see that in case (1) 
we get the symmetric pair (iii) and in case (2) we get the symmetric pair (ii). □ 

Corollary 3.1.2. The pair (GL n+ k,GL n x GLk) is good. 

3.2. All the descendants of the pair (GL n+ k,GL n x GLk) are regular. 

Clearly if a pair (G, H, 9) is regular then the pair (Ge/f, He/F: @) is also regular for any field extension 
E/F. Therefore by Theorem 13 . 1.11 and Theorem l2.4.11l it is enough to prove that the pair {GL n j r i i ^ GL n x 
GL k ,6 n ^ k ) is regular. 

In case n ^ k it follows from the definition since any admissible g 6 GL n+ k lies in GL n x GLk- 
So we can assume n — k > 0. Hence by proposition 12.4.91 it is enough to prove the following key 
lemma. 

Lemma 3.2.1 (Key lemma). Let x E gl^ni^) be a nilpotent element and d := d(x). Then 

Tr(ad(d)|( 3i7i(i? ) xgiii ( F)) J < 2n 2 . 

We will need the following definition and lemmas. 

Definition 3.2.2. We fix a grading on sl2(F) given by h € sl2{F)o and e, / £ sl2(F)\ where (e, h, /) is 
the standard sl2-triple. A graded representation of SI2 is a representation of SI2 on a graded vector 
space V — Vq © V\ such that sl2{F)i(Vj) C Vi+j where i,j 6 Z/2Z. 

The following lemma is standard. 

Lemma 3.2.3. 

(i) Every irreducible graded representation of SI2 is irreducible (-as a usual representation of s^- 

(ii) Every irreducible representation V of SI2 admits exactly two gradings. In one grading the highest 
weight vector lies in Vo and in the other grading it lies in V\ . 

Lemma 3.2.4. ^Let V\ and V2 be two irreducible graded representations of an sl2-triple (e, h, /). Letri := 
dimVi and let Wi £ { — 1, +1} be the parity of the highest weight vectors of Vi- Consider Hom{{V\ 1 V2) e )o 
- the even part of the space of e-equivariant linear maps Vi — ► Vi- Let 



Then 



m := ^{h\(Ho m (V l y 2 Y) Q ) + T^(h\(Hom{V 2 ,Vi) e ) ) - r l r 2- 

— min(ri,r2), r\ ^ r2 mod 2; 

— min(ri, r2)(l + ^1^2)1 T\ = ^2 = mod 2; 

— min(ri,r2) + W\W2 (max(ri, r2) — 1), r\=ri = \ mod 2; 

This lemma follows by a direct computation from the following straightforward lemma. 

Lemma 3.2.5. Let V™ be the irreducible graded representation of SI2 with highest weight A and highest 
weight vector of parity w G {— 1, +1}. Then 

(1) TWIwrfc) = A ^ 



(2) (v?y = v™ { - 

(3) v^®v™; = vr l> 



min(Ai,A2) 

r 'Ai+A2-2i 



i=Q 



^This lemma is similar to Lemma 3.1 in IJRI . section 3.2. The proofs are also similar. 
2 This lemma is similar to Lemma 3.2 in IJRI but computes a different quantity. 
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Proof of the key lemma. Let Vq := V\ :— F n . Let V := Vq ® V\ be a Z/2Z graded vector space. We 
consider ghn(F) as the Z/2Z graded Lie algebra End(V). Note that gl n (F) x gl n {F) is the even part of 
End(V) with respect to this grading. Consider as a graded representation of the s?2 triple (a;,d, 
Decompose V to graded irreducible representations Wi- Let r*j := dimWi and = ±1 be the parity of 
the highest weight vector of Wj. Note that if r, is even then dim(Wi l~l Vo) = dim(W / i PI Vi). If r, is odd 
then dim(Wi C\Vo) = dim(Wi D V%) + Wi. Since dimVb = dimVi, we get that the number of indexes i 
such that ri is odd and w% = 1 is equal to the number of indexes i such that r, is odd and it>j = —1. We 
denote this number by Now 

Tr(ad(d)\(gi n iF)xgi n (F)) x ) ~ 2 ™ 2 = Tr(d\ {Hom{vy y )o ) - 2n 2 = 1/2 ^my, 

where 

mjj := Tr(d|( i f om (i Vi)W - j )a.j ) + Tr(d|( ffom ( W , i,w r i) !, ')o) _ r * r j- 
The TOij can be computed using the previous lemma. 

As we see from the lemma, if r, or rj are even then rriij is non-positive and mu is negative. Therefore, 
if all Ti are even then we are done. Otherwise I > and we can assume that all r% are odd. Reorder the 
spaces Wi such that Wi = 1 for i < I and Wi — — 1 for i > I. Now 

X) TO «= £ (ln-r,-|-l)+ ^ (In -r 3 | - 1) - ^ (n + ^-l)- £ (n + r, - 1) = 

i,j i< I, j < I i>l,j>l i<l,j>l i> I, j <l 

i<l,j<l i>l,j>l i<l,j>l i>l,j<l 

< ( r i + r i)+ £ ( r * + ^)- Z ( r '+ r j)- S (n + r 3 ) = 0. 

i<l,j<l i>l,j>l i<l,j>l i>l,j<l 

□ 
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